TITLE OF THE INVENTION 

METHOD AND APPARATUS FOR IMAGE SENSING 

5 BACKGROUND OF THE INVENTION 

The present invention relates to an image sensing 
apparatus comprising an electronic anti-vibration system 
which performs vibration correction by electrically 

10 extracting a sensed image based on vibration data of the 
image sensing apparatus main body. Particularly, the 
present invention relates to an image sensing method and 
apparatus capable of sensing a moving image and a still 
image, and a storage medium storing control programs for 

15 controlling the image sensing apparatus. 

Conventional image sensing apparatuses such as a 
video, camera or the like have achieved automatic 
functions and multi-functions in various aspects, e.g., 
automatic exposure (AE) , automatic focus (AF) and so 

20 forth. By virtue of this, high quality image sensing is 
possible . 

Furthermore, as the sizes of image sensing 
apparatuses become smaller and magnifications of optical 
systems become higher, vibration of the image sensing 
25 apparatus causes deteriorated sensed image quality. To 
compensate for such influence of vibration of the image 
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sensing apparatus, various image sensing apparatuses 
having a function for correcting vibration have been 
proposed. 

Fig. 1 is a block diagram showing a construction 
5 of a conventional image sensing apparatus having a 
vibration correction function. 

The image sensing apparatus having the vibration 
correction function shown in Fig. 1 comprises an image 
sensing unit 401, a vibration correction section 402, a 
10 signal processor 403, a recording section 404, a 
vibration detector 405, a DC cut filter 406, an 
amplifier 407, a correction variable calculator 408, a 
read-out controller 409, and a timing generator 410. 

The image sensing unit 401 comprises a lens 401a, 
15 and an image sensing device 401b such as a CCD or the 
like . 

The vibration correction section 402 is provided 
to correct image signals, obtained by the image sensing 
device 401b, in order to reduce vibration components of 

20 a moving image influenced by unsteady hands or the like. 
The vibration correction section 402 comprises a first 
delay section 402a serving as delay means for one pixel, 
a first adder 402b, a second delay section 402c serving 
as delay means for one image line, and a second adder 

25 402d. 

The signal processor 403 converts electric signals, 
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where vibration components are reduced by the vibration 
correction section 402, to standard video signals such 
as NTSC signals or the like. The recorder 404, 
comprising a video tape recorder or the like, records 
5 the standard video signals converted by the signal 
processor 403 as image signals. 

The vibration detector 405 comprises an angular 
velocity sensor such as a vibration gyro sensor or the 
like and is installed in the image sensing apparatus 
S 10 main body. 

HJ The DC cut filter 406 cuts off the DC components 

Iff of the angular velocity signal, outputted by the 

J5 vibration detector 405, and passes AC components only, 

5 i.e., vibration components. The DC cut filter 406 may 

n\ 15 employ a high-pass filter (hereinafter referred to as 

^ HPF) for cutting off signals of a predetermined band. 

~f The amplifier 407 amplifies the angular velocity 

signal outputted by the DC cut filter 406 to an 
appropriate level . 
20 The correction variable calculator 408 comprises, 

e.g., a microcomputer or the like, which includes an A/D 
converter 408a, high-pass filter (HPF) 408b, integrator 
408c, and pan/tilt determination section 408d. 

The A/D converter 408a converts the angular 
25 velocity signal outputted by the amplifier 407 to 
digital signal. 
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The high-pass filter 408b, having a variable cut- 
off frequency, cuts off low frequency components of the 
digital signal outputted by the A/D converter 408a. 

The integrator 408c outputs an angle displacement 
5 signal by integrating the signals (angular velocity 

signal) outputted by the high-pass filter 408b, and has 
the function to vary characteristics in an arbitrary 
frequency band. 

The pan/tilt determination section 408d determines 
10 panning or tilting based on the angular velocity signal, 
the angular velocity signal outputted by the integrator 
408c, and an integrated signal, i.e., angle displacement 
signal, on which integration process has been performed 
on the angular velocity signal. The pan/tilt 
15 determination section 408d performs panning control, 
which will be described later, in accordance with the 
levels of the angular velocity signal and angle 
displacement signal . 

The angle displacement signal obtained by the 
20 correction variable calculator 408 serves as a 

correction target value in the control described later. 

The read-out controller 409 shifts a read-out 
start position of the image sensing device 401b in 
accordance with the vibration correction target value 
25 signal, and controls operation timing of the first delay 
402a and the second delay 402c as well as an addition 
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ratio of the first adder 402b and the second adder 402d. 

The timing generator 410 generates driving pulses 
for the image sensing device 401b, the first delay 402a, 
and the second delay 402c, based on the control data of 
5 the read-out controller 409. Driving pulses are 

generated for the image sensing device 401b according to 
the storage and read operation of the image sensing 
device 401b. For the first delay 402a and the second 
delay 402c, driving pulses which serve as a reference 

10 for delaying operation, and whose phases are coherent 

with the driving pulses of the image sensing device 401b, 
are generated. 

The vibration correction comprises the vibration 
detector 405, DC cut filter 406, amplifier 407, 

15 correction variable calculator 408, read-out controller 
409, and timing generator 410. 

Next, operation of the pan/ tilt determination 
section 408d of the correction variable calculator 408 
will be described in detail. 

20 The pan/ tilt determination section 408d determines 

an angular velocity signal from the A/D converter 408a 
and an angle displacement signal from the integrator 
408c. If the angle displacement signal, which is 
obtained by integrating the angular velocity signal, is 

25 larger than the predetermined threshold value, the 

section 408d determines that panning/ tilting operation 
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is being performed, regardless of whether or not the 
angular velocity is larger or smaller than a 
predetermined threshold value. In such case, the low 
cut-off frequency of the high-pass filter 408b is 
5 shifted higher to change the characteristics so that the 
vibration correction system does not respond to low 
frequencies. Further, in order to gradually shift a 
correction position of the image correction means to the 
center of the correction range, control is performed 

10 (hereinafter referred to as panning control) such that a 
time constant serving as an integration characteristic 
of the integrator 408c is reduced, so that an integral 
value by the integrator 408c becomes to a reference 
value (a value obtained if vibration is not detected) . 

15 Note that during the above operation, the angular 

velocity signals and angle displacement signals continue 
to be detected. When the panning or tilting is completed, 
the low cut-off frequency is reduced to expand the 
vibration correction range, then the panning control 

20 ends . 

The panning determination operation will be 
described with reference to the flowchart in Fig. 2. 

First in step S501, the A/D converter 408a 
converts (A/D conversion) an angular velocity signal 
25 amplified by the amplifier 407, to a digital value which 
can be operable by the correction variable calculator 
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408. Next in step S502, the high-pass filter 408b 
operates using the cut-off frequency value prepared in 
the previous step. Then in step S503, the integrator 
408c executes an integration calculation using the time 
5 constant value prepared in the previous step. In step 

S504, the integration result obtained in step S503, i.e., 
an angle displacement signal, is converted from a 
digital value to an analogue value then outputted. 

In step S505, it is determined whether or not the 

10 angular velocity signal is equal to or larger than a 

predetermined threshold value. If the angular velocity 
signal is equal to or larger than the predetermined 
threshold value, determination is made that a 
panning/ tilting operation is being performed. Then in 

15 step S506, the cut-off frequency value f c used for the 

operation of the high-pass filter 408b is increased from 
the current value so that the signal in low frequency 
bands is attenuated in power. Next in step S507, the 
time constant value used in the integration is reduced 

20 from the current value by a predetermined value so that 
the angle displacement output becomes close to the 
reference value. Then, the control operation ends. 

Meanwhile, if the angular velocity signal is not 
equal to or larger than the predetermined threshold 

25 value in step S505, it is determined in step S508 
whether or not the integrated value is equal to or 
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larger than a predetermined threshold value. If the 
integrated value is equal to or larger than the 
predetermined threshold value, determination is made 
that panning or tilting operation is being performed, 
5 and the control proceeds to step S506. Meanwhile, if the 
integrated value is not equal to or larger than the 
predetermined threshold value in step S508, 
determination is made that the camera is in a normal 
control state or panning/tilting operation has been 

10 completed. Then, the control proceeds to step S509 . 

In step S509, the cut-off frequency value used in 
operation of the high-pass filter 408b is reduced from 
the current value by a predetermined value so that the 
signal power in low frequency signals is more decreased. 

15 Next in step S510, the time constant value used in 

integration is increased from the current value by a 
predetermined value so that the integration are more 
effective. Then, the control operation ends. 

By the above described control, the overflow of 

20 integration value exceeding a correction target value is 
prevented. As a result, the correction target value is 
kept stationary, and a stable ant i -vibration control is 
achieved. 

Next, the correction means in the above-described 
25 conventional example will be briefly described with 
reference to Fig. 3. 
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Referring to Fig. 3, reference numeral 601 denotes 
an entire image sensing area of the image sensing device 
401b; 602, an extraction frame set in the entire image 
sensing area 601, where an image signal is actually 
5 converted to standard video signals and outputted; and 
603, an object sensed by the image sensing device. 

Standard video signal in this stage is seen as an 
image indicated as reference numeral 604. More 
specifically, reference numeral 604 denotes an image 
^ 10 area on a monitor where the video signal is reproduced, 

Jjf and 603' denotes the object reproduced in the image area 

Ul 604 of the monitor. By extracting an image signal from a 

LP 

HF sensed image as will be described later, the periphery 

s is removed from the entire image sensing area 601 and 

flj 15 the remaining portion, that is an extracted image is 

nl outputted as standard video signals. By this, the image 

area 604 is reproduced on the monitor. 

Fig. 3 shows a case where an operator, who 
performs image sensing of the object 603, shifts the 
20 image sensing apparatus in a lower left direction 
indicated by arrows 605, 605', and 605". 

In this state, if an image is extracted by using 
an extraction frame 602' which corresponds to the 
extraction frame 602, the obtained video signal would 
25 represent the object 603, which is shifted by the amount 
of vector indicated by the arrow 606. It is possible to 
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obtain the image 604 if the extraction position is 
corrected and moved from the extraction frame 602' to 
the extraction frame 602", using an image displacement 
607 obtained based on a vibration amount of the image 
sensing apparatus, i.e., vibration correction target 
value. By employing such principle, image vibration 
correction is realized. 

Next, extraction of an image sensing area for 
correction will be described with reference to Fig. 4. 

In Fig. 4, reference numeral 701 denotes the 
entire image sensing device; and 702, pixels 
constructing the entire image sensing device 701, each 
serving as a photoelectric converter. Based on electric 
driving pulses generated by a timing generator (not 
shown) , charge storage and read-out controls are 
performed pixel by pixel. Reference numerals 703 and 704 
are extraction frames similar to the extraction frame 
602 shown in Fig. 3. This will be explained for a case 
where video signals are extracted with, e.g., the 
extraction frame 703 . 

First, starting from the pixel "S", 
photoelectrically converted charges are read in the 
sequence of the arrow 706. The reading operation starts 
in concurrence with the synchronization period of output 
video signals. During the synchronization period, 
reading is performed at a faster transfer rate than the 
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normal reading rate until the reading-out reaches the 
pixel preceding the pixel "A" by one pixel . 

After the synchronization period, charges for a 
real image period are read starting from the pixel W A" 
5 to the pixel W F" at the normal reading rate, as a line 
of image data in video signals. 

During the horizontal synchronization period, 
charges of pixels are read from the pixel W F" to the 
pixel before the pixel W G" at a faster transfer rate 
O 10 than the normal reading rate, preparing for reading the 

ry next image period. Starting from the pixel U G" , reading 

ui 

In is performed in a similar manner to the reading from the 

^ pixel "A" to the pixel XV F". 

J" By controlling the reading operation as described 

Sr: 15 above, for instance, the central part of the image 

sensing device 401b can be selectively extracted from 
tfl the entire image sensing area 601 of the image sensing 

device 401b, and video signals can be obtained. 

Described next with reference to Fig. 4 is moving 
20 the extraction position, necessary in a case where the 

image sensing surface is shifted due to vibration on the 
image sensing apparatus as described in Fig. 3. 

In a case where the movement of the object 
(vibration of the image sensing apparatus) , which is 
25 equivalent to the arrow 705, is detected on the image 

sensing device 401b, the extraction frame 703 is changed 
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to the extraction frame 704 so that an image without any 
object movement is obtained. 

In order to change the extraction position, the 
aforementioned read-out start position is changed from 
5 the pixel "A" to the pixel "B" . By this change, it is 
possible to selectively extract a partial image from the 
entire image sensing area 601 of the image sensing 
device 401b, similar to the case where reading starts 
from the pixel "A" . 

10 More specifically, photoelectrically converted 

charges are read from the pixel "S" iri the sequence of 
the arrow 706, as similar to the case of reading the 
extraction frame 703 . The reading operation starts in 
concurrence with the synchronization period of output 

15 video signals. During the synchronization period, 

reading is performed at a faster transfer rate than the 
normal reading rate until the reading reaches the pixel 
one-pixel prior to the pixel "B" . Then, in the real 
image period, reading starts from the pixel XX B" . 

20 As described above, a partial image sensing area, 

which is the periphery of the image sensing device 401b, 
is read in advance for the amount corresponding to 
vibration correction data during the synchronization 
signal period which does not appear in the real image 

25 period, and the part of the image sensing device 401b is 
selectively read out based on the vibration data. By 
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this, it is possible to obtain video signals from which 
displacement caused by vibration on the image sensing 
apparatus is removed. 

Next, pixel shifting operation by the first delay 
5 402a, second delay 402c, first adder 402b, and second 
adder 402d will be described in detail with reference to 
Fig. 5. 

Fig. 5 is a block diagram showing a construction 
of an image sensing apparatus for performing finer image 
p 10 correction by extracting an image from the image sensing 

fy device 401b. Components shown in Fig. 5, which are 

jfji identical to that of the above-described conventional 

]p example in Fig. 1, have the same reference numerals. 

Referring to Fig. 5, the first adder 402b 
Zr! 15 comprises a first multiplier 801, a second multiplier 

^ 802, an adder 803, and addition controller 804. The 

second adder 402d comprises a first multiplier 805, a 
second multiplier 806, an adder 807, and addition 
controller 808. 

20 The image extraction, as described with reference 

to Fig. 4 can only be performed in each unit pixel 702 
of the image sensing device 401b. Therefore, the 
correction could not be made for a shift less than one 
pixel pitch. Therefore, the first delay 402a, second 

25 delay 402c, first adder 402b, and second adder 402d 
perform a fine pixel shifting which accompanies a 
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correction calculation for an amount less than one pixel 
unit . 

Referring to Fig. 5, reference numeral 809 denotes 
an input terminal for inputting a sensed image signal 
5 obtained by the image sensing device 401b. While the 

sensed image signal is inputted to the second multiplier 
802 of the first adder 402b, the signal is inputted to 
the first multiplier 801 through the first delay 402a 
constructed by a CCD or the like. After the first 
Q 10 multiplier 801 and the second multiplier 802 multiply 

Pj the respective image signal by a predetermined 

in 

y] multiplier, addition is executed by the adder 803. As a 

III 

Jcf result, a horizontal pixel-shifting correction has been 

J~ performed on the sensed image signal. 

5= 15 The sensed image signal, on which horizontal 

~; pixel-shifting correction has been performed, is 

"if inputted to the second multiplier 806 of the second 

adder 402d, and to the first multiplier 805 through the 
second delay 402c, which is constructed by a CCD or the 
20 like. The first multiplier 805 and the second multiplier 
806 multiply the signals by predetermined multipliers. 
The adder 807 adds the outputs of the multipliers 805 
and 806. Thus, the sensed image signal on which 
horizontal and vertical pixel-shifting correction has 
25 been performed is obtained. The image signals are 

encoded to video signals by the signal processor 403 in 



- 14 - 




the next step. 

The read-out controller 409 controls the timing of 
driving pulses generated by the timing generator 410 for 
performing image extraction control of the image sensing 
5 device 401b based on the correction target value signal 
which corresponds to vibration of the image sensing 
apparatus, which is generated by the correction variable 
calculator 408 shown in Fig. 1. Meanwhile, the read-out 
controller 409 controls the addition controller 804 of 
10 the first adder 402b and the addition controller 808 of 
the second adder 402d so as to perform addition 
operation at an appropriate addition ratio. 

This addition operation will now be described in 
detail with reference to Figs. 6A, 6B, 7A and 7B. 
15 Figs. 6A and 6B are explanatory views showing the 

process of the first delay 402a and the first adder 402b 
in unit of the pixels 702 of the image sensing device 
401b. The pixels 702 of the image sensing device 401b 
are arranged in a regular order in the vertical and 
20 horizontal directions on the surface of the image 

sensing device 401b as shown in Fig. 4. Figs. 6A and 6B 
only show the pixels arranged in the horizontal 
direction for explanatory purpose. 

First, addition operation performed in pixel units 
25 shown in Fig. 6A is described. 

A horizontal center position of the n-th pixel of 
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the pixels 702 of the image sensing device 401b is 
referred to as a horizontal pixel center 901. Similarly, 
a horizontal center position of the (n+l)th pixel of the 
pixels 7 02 is referred to as a horizontal pixel center 
5 902. Fig. 6A graphically shows a method of calculating 
an imaginary center of the n-th pixel and the (n+l)th 
pixel. The n-th pixel value which is multiplied by 1/2, 
denoted by numeral 903, the (n+l)th pixel value which is 
multiplied by 1/2, denoted by 904, are added, thereby 

10 obtaining a pixel value at the imaginary center 905 of 

the n-th pixel and (n+l)th pixel. Similarly, a value in 
which the (n+l)th pixel is multiplied by 1/2 is added to 
a value in which the (n+2)th pixel is multiplied by 1/2, 
thereby obtaining a pixel value at the imaginary center 

15 of the (n+l)th pixel and the (n+2)th pixel. 

Next, another addition operation shown in Fig. 6B 
will be described. Fig. 6B adopts a different addition 
ratio from that of Fig. 6A, showing a case of executing 
an addition with a ratio of 7/10 to 3/10. 

20 In Fig. 6B, as similar to Fig. 6A, the horizontal 

center position of the n-th pixel of the pixels 702 of 
the image sensing device 401b is defined as a horizontal 
pixel center 901, and the horizontal center position of 
the (n+l)th pixel of the pixels 702 of the image sensing 

25 device 401b is defined as a horizontal pixel center 902. 
In order to obtain an image having 7/10 of the n-th 
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pixel value and 3/10 of the (n+l)th pixel value, a pixel 
value 903' in which the n-th pixel is multiplied by 7/10 
is added to a value 904' in which the (n+l)th pixel is 
multiplied by 3/10, thereby obtaining an image 905' 
having 7/10 of the n-th pixel and 3/10 of the (n+l)th 
pixel. Similarly, by adding a value in which the (n+l)th 
pixel is multiplied by 7/10 to a value in which the 
(n+2)th pixel is multiplied by 3/10, it is possible to 
obtain an pixel having 7/10 of the (n+l)th pixel and 
3/10 of the (n+2)th pixel. 

As described above, by adjusting the addition 
ratio of adding the n-th pixel and (n+l)th pixel, it is 
possible to obtain pixel data for an arbitrary position 
of a pixel in the horizontal direction. Note that 
assuming that an n-th pixel value is generally 
multiplied by k, the addition ratio is obtained by 
multiplying an (n+l)-th pixel value by (1 - K) (where, 0 
< K < 1) . Where the multiplication ratio is set to 1:0 
or 0:1, addition is not performed. Then, the normal 
read-out operation is performed. 

Figs . 7A and 7B are explanatory views showing the 
process of the second delay 402c and the second adder 
402d in the horizontal line units of the image sensing 
device 401b . 

First, addition operation performed in horizontal 
line units shown in Fig. 7A will be described. The 
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horizontal line is constructed by a horizontal array of 
pixels of the image sensing device 401b. Vertical center 
position of the n-th line is referred to as a vertical 
pixel center 1001. Similarly, the vertical center 
position of the (n+l)th line is referred to as a 
vertical pixel center 1002 . 

Fig. 7A graphically shows a method of calculating 
a pixel value at the center positioned between the n-th 
line and the (n+l)th line. A value 1003 in which the n- 
th line is multiplied by 1/2 is added to a value 1004 in 
which the (n+l)th line is multiplied by 1/2, thereby 
obtaining a line 1005. Similarly, a value in which the 
(n+l)th line is multiplied by 1/2 is added to a value in 
which the (n+2)th line is multiplied by 1/2, thereby 
obtaining the center image of the (n+l)th line and 
(n+2)th line. 

Next, addition operation performed in horizontal 
line units shown in Fig. 7B will be described. Fig. 7B 
adopts a different addition ratio from that of Fig. 7A, 
showing a case of executing addition with a ratio of 
7/10 to 3/10. 

In Fig. 7B, similar to Fig. 7A, the vertical 
center position of the n-th line of the pixels 702 of 
the image sensing device 401b is referred to as a 
vertical pixel center 1001, and the vertical center . 
position of the (n+l)th line is referred to as the 



- 18 - 



# 



Us? 



m 



vertical pixel center 1002. In order to obtain an image 
having 7/10 of the n-th line and 3/10 of the (n+l)th 
line, a value 1003' in which the n-th line is multiplied 
by 7/10 is added to a value 1004' in which the (n+l)th 
5 line is multiplied by 3/10, thereby obtaining a line 

image 1005' having 7/10 of the n-th line and 3/10 of the 
(n+l)th line. Similarly, by adding a value in which the 
(n+l)th line is multiplied by 7/10 to a value in which 
the (n+2)th line is multiplied by 3/10, it is possible 

10 to obtain an image having 7/10 of the (n+l)th line and 
3/10 of the (n+2)th line. 

As described above, by adjusting the addition 
ratio of adding the n-th line and the (n+l)th line, it 
is possible to obtain line data for an arbitrary 

15 position in one line in the vertical direction. 

Note that assuming that the n-th line is generally 
multiplied by k, the addition ratio is obtained by 
multiplying the (n+l)th line by (1-k) (0 < K < 1) . If 
the multiplication ratio is set to 1:0 or 0:1, addition 

20 is not performed. Thus, the normal read-out operation is 
performed. 

As set forth above, correction operation in pixel 
units is achieved by reading out data by the image 
sensing device 401b. Correction operation for an amount 
25 less than one pixel unit is achieved by performing pixel 
shifting. By this, excellent anti-vibration can be 
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attained. 

However, recently an image sensing apparatus 
sensing both a moving image and a still image has been 
proposed. Such image sensing apparatus also employs a 
5 similar ant i -vibration system to the above. However, the 
anti-vibration system employed by the foregoing 
conventional image sensing apparatus realizes anti- 
vibration process by pixel shifting operation. In this 
case, data stored between pixels are added up, resulting 
10 in deteriorated resolution in images. Particularly in a 
case of sensing a still image wherein a correction 
between frames by image extraction is not important, a 
problem of deteriorating resolution in images 
(hereinafter referred to as a first problem) rather 
15 arises . 

Next, a second problem of the conventional 
apparatus will be described. 

As a conventional method of anti -vibration, the 
method disclosed in Japanese Patent Application Laid- 
20 Open No. 60-143330 is known. 

The conventional method includes an optically and 
electronically vibration correction methods. According 
to the optical correction method, vibration is detected 
from an image sensing apparatus main body by a 
25 rotational gyro sensor, and based on the detected result, 
the optical systems including lens and the image sensing 
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apparatus main body are controlled. According to the 
electronic correction method, signal transferring of an 
image sensing device is divided into a high-velocity 
transfer mode and a normal transfer mode, and the number 
5 of pixels transferred in high-velocity transferring is 
controlled. The above -de scribed conventional examples 
employs the electronic correction method. The electronic 
correction method particularly provides an advantage of 
apparatus size reduction, 
p 10 In the electronic anti-vibration method, the 

• rv 

nj number of pixels transferred is controlled. In a case of 

yi employing, an image sensing device reading vertically 

jf neighboring pixels simultaneously, the pixel area where 

J" signals are normally transferred is shifted at two pixel 

JT» 15 pitch. This limits a resolving power in anti -vibration . 

Lr: For a method of improving the resolving power in 

^jf anti-vibration, the method disclosed in Japanese Patent 

Application Laid-Open No. 3-132173 is known. According 
to this method, vertically neighboring two pixels which 
20 are read simultaneously by an image sensing device are 

shifted at one pixel pitch. Since pixels are shifted at 
one pixel pitch within an image area where signals are 
normally transferred, smooth pixel movement is achieved. 
For instance, in a case of reading pixels, the 
25 conventional art reads pixel data at two pixel pitch as 
shown in Fig. 8A. When shifting read-out data, the 
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conventional apparatus, instead of shifting pixels at 
two pixel pitch as shown in Fig. 8B, shifts image data 
at one pixel pitch, as shown in Fig. 8C. By this, a 
pixel area normally transferred can be shifted at one 
5 pixel pitch. In this, regard, since the pixel 

combinations are different, color process has to be 
changed later to perform color processing according to 
the combination of pixels. 

Note that in Figs. 8A, 8B and 8C, Y denotes an 
Q 10 yellow color filter; C, a cyan color filter; M, a 

Hi magenta color filter; and G, a green color filter, 

Li jj 

[Jl Meanwhile, there is a pixel shifting technique 

available today in which image signals of the vertically 
neighboring two pixels are added in a predetermined 
15 ratio, and an image is shifted by changing the ratio, 

thereby improving a resolving power in anti -vibration . 
Herein, pixel shifting will be described with reference 
to Fig. 9. 

Referring to Fig. 9, first, an image signal for 
20 one horizontal scan period (H) is stored in 1-H delay 
memory 1701. A first multiplier 1702 multiplies 
currently inputted image signal by 1^, A second 
multiplier 1703 multiplies the outputs of the 1-H delay 
memory 17 01 by (1-1^) . These multiplied values are added 
25 by an adder 1704. By this, when is 1, currently 

inputted image signal is outputted. When is 1/2, an 
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intermediate image signal between the currently inputted 
image signal and the signal in the 1-H delay memory 17 01 
is outputted. When Ky is 0, an image signal which is 1H 
prior to what is stored in the 1-H delay memory 1701 is 
5 outputted. By virtue of this, it is possible to shift 
positions of image signals. If the addition ratio is 
adjusted in small increments, smaller vibration can be 
accommodated . 

However, in the above-described conventional 
4f 10 example in Fig. 9, if anti-vibration operation is 

"J performed by shifting a pixel area where signals are 

in normally transferred at one pixel pitch, the resolving 

jp power is not high enough to perform anti-vibration. As a 

5 result, very small vibration remains, or excessive 

fy 15 vibration is caused on the contrary. Moreover, 

?%i performing ant i -vibration by pixel shifting causes a 

^ problem of "resolution unevenness " and flickers on a 

screen. Herein, the "resolution unevenness" will be 
described; 

20 Since pixel shifting is performed by adding image 

signals of the vertically neighboring two pixels as 
described above, the resolution is deteriorated. Shown 
in Fig. 10 is a graph showing deterioration of 
resolution. The ordinate in Fig. 10 indicates an image 

25 position, and the abscissa indicates resolution. 

Reference numerals 1801 to 1805 denote image signals 
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respectively . 

As can be seen from Fig, 10, the highest 
resolution is achieved when pixel shifting is not 
performed. The resolution falls to the lowest level when 
5 pixels are shifted to an intermediate position of two 
pixels. Therefore, if pixel shifting to an arbitrary- 
position is performed in anti-vibration control, 
resolution changes one after another, causing flickers 
on a screen. This is called "resolution unevenness." 
Q 10 The occurrence of the resolution unevenness is the 

fU second problem to be solved in the conventional art. 

Cri SUMMARY OF THE INVENTION 

~ 15 The present invention is made in consideration of 

£\ the above situation, and a first object of the present 

jjf= invention is to provide an image sensing method and 

^ apparatus capable of performing vibration correction 

similar to the conventional example when sensing a 
20 moving image, and obtaining a sensed image with no image 
deterioration when sensing a still image, so as to solve 
the above-described first problem. 

A second object of the present invention is to 
provide an image sensing method and an apparatus capable 
25 of eliminating the resolution unevenness while securing 
a resolving power necessary for ant i -vibration, so as to 
solve the above -described second problem. 
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Other features and advantages of the present 
invention will be apparent from the following 
description taken in conjunction with the accompanying 
drawings, in which like reference characters designate 
5 the same or similar parts throughout the figures thereof. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The accompanying drawings, which are incorporated 
10 in and constitute a part of the specification, 
Q illustrate embodiments of the invention, and together 

rj with the description, serve to explain the principles of 

LFj the invention . 

yi 

if Fig. 1 is a block diagram showing a construction 

™ 15 of a conventional image sensing apparatus; 

Fig. 2 is a flowchart showing operation steps of 
L?: panning control performed by the conventional image 

^ sensing apparatus ; 

Fig. 3 is an explanatory view for briefly 
20 explaining vibration correction means of the 
conventional image sensing apparatus ; 

Fig. 4 is an explanatory view for explaining 
extraction of a stored image by vibration correction 
means of the conventional image sensing apparatus; 
25 Fig. 5 is a block diagram for explaining pixel 

shifting in the conventional image sensing apparatus; 

Figs . 6A and 6B are explanatory views explaining 
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horizontal pixel shifting in the conventional image 
sensing apparatus; 

Figs . 7A and 7B are explanatory views explaining 
vertical pixel shifting in the conventional image 
5 sensing apparatus; 

Figs . 8A to 8C are explanatory views for 
explaining brief operation of vibration correction means 
in the conventional image sensing apparatus; 

Fig. 9 is a block diagram showing a construction 
O 10 of a section performing pixel shifting in the 

Hi conventional image sensing apparatus; 

: =5 
: s t 

" % 

Ul Fig. 10 is a graph showing relationship between an 

m 

£ image position upon pixel shifting and resolution, in 

the conventional image sensing apparatus; 
15 Fig. 11 is a block diagram showing a construction 

of an image sensing apparatus according to a first 
embodiment of the present invention; 

Fig. 12 is a flowchart showing operation steps of 
pixel shifting control performed by the image sensing 
20 apparatus according to the first embodiment of the 
present invention ; 

Fig. 13 is a block diagram showing a construction 
of an image sensing apparatus according to a modified 
example of the first embodiment; 
25 Fig. 14 is a block diagram showing a construction 

of an image sensing apparatus according to a second 



n 
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embodiment of the present invention; 

Figs. 15A and 15B are explanatory views for 
explaining an image area extracted by electronic anti- 
vibration control in the image sensing apparatus 
5 according to the second embodiment; 

Fig. 16 is a flowchart showing operation flow of 
the image sensing apparatus according to the second 
embodiment; 

Fig. 17 is a flowchart showing operation flow of 
10 the image sensing apparatus according to the second 
embodiment; 

Fig. 18 illustrates a position (H 0 , V 0 ) where an 
image extraction (image shifting) is started with 
respect to the position (H c/ V c ) without image shifting; 
15 Fig. 19 illustrates how interpolated pixels are 

generated from image signals ; 

Fig. 2 0 is a graph for explaining a process of 
regenerating correction variable data in the image 
sensing apparatus according to the second embodiment; 
20 Fig. 21 illustrates how interpolated pixels are 

quantized so that they are equally spaced; and 

Fig. 22 illustrates how interpolated pixels are 
generated where a resolving power is raised to three 
times . 

25 

DETAILED DESCRIPTION OF THE INVENTION 
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Preferred embodiments of the present invention 
will be described in detail in accordance with the 
accompanying drawings . 

<First Embodiment> 

First, the first embodiment is described with 
reference to Figs . 11 and 12 . 

Fig. 11 is a block diagram showing a construction 
of an image sensing apparatus according to the first 
embodiment. Components shown in Fig. 11, which are 
identical to that of the above-described conventional 
example in Fig. 1, have the same reference numerals. 

Fig. 11 differs from Fig. 1 by having an image 
sensing mode switch 411 in addition to the construction 
shown in Fig. 1. The image sensing mode switch 411 
allows users to switch the record mode of the recording 
section 404, between a moving image recording mode and a 
still image recording mode. The recording section 404 
employs a record format corresponding to the selected 
image sensing mode, depending on a recording medium. 

In the above -described conventional example, the 
read-out controller 409 respectively controls the timing 
generator 410 and the first and second adders 402b and 
402d based on vibration correction target value data 
obtained by the correction variable calculator 408. 
However, in the present embodiment, the read-out 
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controller 409 changes operation of the timing generator 
410 and the first and second adders 402b and 402d in 
accordance with the state of the image sensing mode 
switch 411. 

The operation of the image sensing mode switch 411 
is now described with reference to the flowchart in Fig. 
12. 

First in step S201, a vibration correction target 
value is obtained by the correction variable calculator 
408 based on vibration data obtained by the vibration 
detector 405. The read-out controller 409 controls read- 
out operation of the image sensing device 401b so as to 
perform an anti -vibration correction according to the 
obtained vibration correction target value. Next in step 
S202, it is determined whether the image sensing mode 
switch 411 is set in the moving image sensing mode or in 
the still image sensing mode. In a case where it is set 
in the still image sensing mode, it is determined in 
step S203 whether or not the record mode of the 
recording section 404 is ready for a still image, i.e., 
set in the still image recording mode. If the recording 
section 404 is set in the still image recording mode, 
the process proceeds directly to step S2 04, while if the 
recording section 404 is set in the moving image 
recording mode, the record mode of the recording section 
404 is changed from the moving image recording mode to 
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the still image recording mode in step S206, then the 
control proceeds to step S204. 

In step S204, addition control is performed by- 
setting the value K of the adders 402b and 402d to 1 so 
as not to perform fine pixel-shifting operation, based 
on the above-described reason. Then, the control 
proceeds to step S2 05 where it is determined whether or 
not an image sensing start switch (not shown) has been 
depressed. In a case where the image sensing start 
switch is not depressed, the process ends without 
further processing. In a case where the image sensing 
start switch is depressed, still image sensing is 
executed in step S207 and the control operation ends. 

Meanwhile, in a case where the image sensing mode 
switch 411 is set in the moving image sensing mode in 
step S202, it is determined in step S208 whether or not 
the record mode of the recording section 404 is ready 
for a moving image, i.e., set in the moving image 
recording mode. If the recording section 404 is set in 
the moving image recording mode, the process proceeds 
di rectly to step S2 09, while if the recording section 
404 is set in the still image recording mode, the record 
mode of the recording section 404 is changed from the 
still image recording mode to the moving image recording 
mode in step S211, then the control proceeds to step 
S209. 
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In step S209, as similar to the conventional 
example, addition control for fine pixel-shifting 
operation is performed by calculating an addition ratio 
value K for the adders 402b and 402d, in order to 
5 perform correction for an amount less than one pixel 
based on the correction target value data. Then, the 
control proceeds to step S210 where it is determined 
whether or not an image sensing start switch (not shown) 
has been depressed. In a case where the image sensing 
O 10 start switch is not depressed, the process ends without 

fU further processing. In a case where the image sensing 

UPf start switch is depressed, moving image sensing is 

jj executed in step S212 and the control operation* ends . 

By the above operation, the fine pixel-shifting 
^ 15 operation is performed when sensing a moving image, as 

IH conventionally performed. As a result, an image where 

^ connections between image fields are smooth is obtained. 

Meanwhile in a case of sensing a still image, since the 
fine pixel shifting is not performed, it is possible to 
20 obtain an image where resolution is not greatly 
deteriorated . 

<Modified Example of First Embodiment> 

25 Next, a modified example of the first embodiment 

will be described with reference to Fig. 13. Fig. 13 is 
a block diagram showing a construction of an image 
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sensing apparatus according to a modified example to the 
first embodiment which will be described later. 
Components shown in Fig. 13, which are identical to that 
of the aforementioned conventional example in Fig. 1 and 
5 that of the first embodiment in Fig. 11, have the same 
reference numerals. 

Fig. 13 differs from Fig. 11 by having a first 
switch 301 and a second switch 302 in addition to the 
construction shown in Fig. 11. The first switch 3 01 and 
□ 10 the second switch 3 02 select a-a' or b-b' in accordance 

fij with the switching mode of the image sensing mode switch 

jjj 411, wherein a-a' is a route in which image signals flow 

J- from the image sensing device 401b to the first adder 

402b, then through the second delay 402c and second 
^ 15 adder 402d to the signal processor 403, while b-b' is a 

^ route in which image signals flow from the image sensing 

O device 401b to the signal processor 403 directly. 

A switch, capable of switching the image signal 
flows may be employed for the first switch 3 01 and the 
20 second switch 302. The switching operation is controlled 
such that the first switch 3 01 and the second switch 3 02 
are switched simultaneously. 

For sensing a moving image, the first switch 301 
and the second switch 3 02 select the a-a' side so that 
25 not only the vibration correction operation which is 

executed by extracting an image from the image sensing 
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device 401b, but also the above -described pixel shifting 
control is performed. Accordingly, smooth ant i -vibration 
operation is performed. 

For sensing a still image, the first switch 3 01 
5 and the second switch 302 select the b-b' side so that 
vibration correction operation, executed by extracting 
an image from the image sensing device 401b, is 
performed without using the block including the first 
delay 402a, the second delay 402c, first adder 402b, and 
O 10 the second adder 402d. Accordingly, a still image where 

nj resolution is not deteriorated can be obtained. 

fjl <Second Embodiment> 

7" 15 Fig. 14 is a block diagram showing a construction 

of an image sensing apparatus (video camera) according 
L?j to the second embodiment of the present invention. In 

^ Fig. 14, reference numeral 101 denotes lenses; 102, an 

image sensing device such as a CCD or the like; 103, an 
20 amplifier; 104, a camera signal processor; 105, an 

enlargement/ interpolation/pixel shifting circuit for 
performing enlargement process, interpolation process, 
and pixel shifting process on image data; 106, a CCD 
driver; 107, a pitch angular velocity sensor; 108; a yaw 
25 angular velocity sensor; 109 and 110, amplifiers; 111, 
an ant i -vibration control microcomputer; and 112, a 
switch. Note that the pitch angle sensor 107 and the yaw 
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angle sensor 108 of the second embodiment correspond to 
the vibration detector 405 of the first embodiment, and 
the anti-vibration control microcomputer 111 of the 
second embodiment correspond to the correction variable 
calculator 408 of the first embodiment. 

An enlargement or magnifying process by the 
circuit 105 is to compensate for the difference in pixel 
number of the CCD 102 between in vertical and horizontal 
directions so that there occurs no difference in 
resolution between in the directions, and increases the 
number of apparent pixels by magnifying the image in the 
less direction in pixel number. The circuit 105 is also 
capable of performing pixel-interpolation process so as 
to generate a pixel at shift position where a shift 
amount of the pixel shift or displacement for anti- 
vibration includes a shift less than one pixel. Further, 
the circuit 105 is capable of shifting a read-out start 
position in the CCD 102. In the second embodiment, since 
the interpolation process is performed accompanied with 
the pixel displacement operation, the pixel shift for 
ant i -vibration includes the interpolation operation and 
the shifting of the read-out start position. Hereafter, 
a pixel shift operation by shifting the read-out start 
position will be referred to as "rough pixel shifting", 
while a pixel shifting in less than one pixel is as 
"fine pixel shifting". 
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Referring to Fig. 14, light from an object passes 
through the lenses 101 and an image is formed on the 
image sensing device 102. The image on the image sensing 
device 102 is photoelectrically converted, then 
5 amplified to an appropriate level by the amplifier 103, 
inputted to the camera signal processor 104, and 
converted to standard video signals. 

The image sensing apparatus shown in Fig. 14 
comprises an electronic- image stabilization function. 
Q 10 Activation or non-activation of this anti-vibration 

nJ function is determined by detecting the state of the 



switch 112 . The pitch and yaw angular velocity sensors 
107 and 108 detect vibration angular velocity of the 
image sensing apparatus. The angular velocity signals 

15 are amplified by the amplifiers 109 and 110, and 

inputted to the anti-vibration control microcomputer 111, 
then integrated and calculated as an angle displacement. 
In accordance with the displacement angle 6 obtained in 
the foregoing manner and a focal point distance f of the 

20 optical system, the pixels (corresponding to f x tanG) 
displaced on the image sensing device 102 due to 
vibration are shifted to the direction opposite to the 
displaced direction, thereby performing vibration 
correction . 

25 Figs. 15A and 15B are explanatory views for 

explaining an image area extracted by electronic anti- 
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vibration control. In Fig. ISA, reference numeral 201 
denotes an effective pixel area of the image sensing 
device 102. The start point (x 0 , y 0 ) of a section 202 
(hereinafter referred to as a partial area 202) of the 
5 effective pixel area is moved in the range of the 

effective pixel area 2 01 in a way that only the partial 
area 202 is extracted and vibration correction is 
performed. Then, the partial area 202 only is displayed 
on a monitor such as a video monitor or a finder or the 
Q 10 like. 

.=3. 

ry As a technique of such displaying operation, the 

in 

\f% first technique is to temporarily store an image for the 

effective pixel area 201 by using a field memory, and 
while reading the image for the partial area 202 from 
y 15 the memory, enlarge the image so as to fit in the area 

53f 203 shown in Fig. 15B and interpolate the 

L J3 horizontal /vertical scan lines. The second technique is 

to employ a large CCD having high density and a large 
number of pixels as the image sensing device 102 so that 
20 the extraction area of the partial area 2 02 satisfies 
the number of scan lines necessary for the standard 
video signals. Both techniques require an expensive 
field memory or a large CCD. Therefore, the second 
embodiment of the present invention utilizes a CCD for 
25 the widely used PAL (phase alternation line) for an 
image sensing apparatus complying with the NTSC 
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(National Television System Committee) . 

The CCD for PAL has a high pixel density in the 
vertical direction. Therefore, with respect to the 
vertical scan direction, an image may be extracted 
5 without further process by using the CCD driver 106, 
e.g., a timing generator or the like. With respect to 
the horizontal scan direction, enlargement process is 
performed in a vertical-horizontal ratio by using a line 
memory. By this, an inexpensive vibration correction 

10 apparatus is realized. 

The image sensing apparatus in Fig. 14 has a 
construction having the above-described correction 
system. When pixel shifting is performed in the vertical 
scan direction, rapid drain control is performed by the 

15 CCD driver 106 to extract a desired scan area. 

Specifically, in the second embodiment, image data is 
read out of the CCD 102, alternately as illustrated in 
Fig. 8B and Fig. 8C . The alternate reading-out causes to 
change pixel combinations, the camera signal processor 

20 104 is controlled such that color process corresponding 
to the pixel arrangement is performed. When pixel 
shifting is performed in the horizontal scan direction, 
a read-out start position of a horizontal scan image, 
stored in the enlargement /interpolation/pixel shifting 

25 circuit 105 where image signals processed by the camera 
signal processor 104 are inputted, is varied in 
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accordance with the amount of pixel shifting for 
vibration correction, and the enlargement and 
interpolation are performed in an appropriate vertical- 
horizontal ratio. With regard to shifting an amount less 
5 than one pixel (hereinafter referred to as a "decimal 
fraction section") , the enlargement/ interpolation/pixel 
shifting circuit 105 can perform fine pixel shifting. 

Ant i -vibration control operation of the image 
sensing apparatus according to the second embodiment 

= 10 will be described with reference to the flowcharts in 

FU Figs. 16 and 17. 

Ul The flowchart in Fig. 16 shows a process of 

J5 calculating an angle displacement by integrating the 

- angular velocity signals detected by the angular 

IXt 15 velocity sensors 107 and 108. This process is a periodic 

interruption process executed by the anti-vibration 
control microcomputer 111, and executed with a frequency 
ten times larger than the field frequency, i.e., 600 Hz 
in a case of the NTSC, in the second embodiment. This 
20 frequency corresponds to a sampling frequency of the 
angular velocity signal i.e., the frequency of the 
calculated angle displacement. The interruption process 
of the ant i -vibration control microcomputer 111 starts 
when, for instance, a counter counting up/down at a 
25 predetermined frequency dividing of an oscillation clock 
coincides with data corresponding to 1/600 sec. Although 
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angular velocity signals are inputted to an A/D 
converter of the ant i -vibration control microcomputer 
111 as described in Fig. 14, for the purpose of simple 
description of the present embodiment, it is assumed 
5 herein that the operation mode of the A/D converter is 
set in a scan mode in which A/D operation is repeated 
all the time. 

Referring to Fig. 16, first in step S3 01, angular 
velocity signals obtained as an A/D sample are subjected 
10 to high-pass filtering process so as to remove an 

influence of DC components. Next in step S302, the 
U] angular velocity signals, in which band area is limited, 

=p are integrated and an angle displacement is calculated. 

5 The calculated angle displacement corresponds to the 

m 15 displacement angle of the image sensing apparatus body, 

ry The processes performed during steps S3 03 and S3 05 are 

/p[ the routine for determining whether or not displacement 

angle calculation is performed ten times in one field. 
More specifically, in step S303, a register m storing 
20 the number of times parameters is incremented by one. In 
step S3 04, it is determined whether or not m=10 is 
satisfied. If m=10 is satisfied, i.e., if the 
interruption process has been performed ten times, m=0 
is set in step S305 and initialization is performed for 
25 the next field. Then, the control operation ends. 

Meanwhile, if m=10 is not satisfied in step S304, 
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i.e., if the interruption process has not been performed 
ten times, the aforementioned step S305 is skipped and 
the control operation ends. 



5 once for each processing period of one field. The 

control steps shown in Fig. 16 are executed ten times in 
the processing period of one field. The control steps in 
Fig. 17 are performed before starting the process for 
the next field, i.e., at the end of the process for the 
10 present field. 



of m=0 is made until m=0 is satisfied. When m=0 is 
satisfied, i.e., the interruption process has been 
performed ten times in the present field and m is 

15 initialized, it is determined in the next step S402 
whether or not anti-vibration function is activated 
based on the state of the switch 112. If the switch 112 
is ON, i.e., ant i -vibration function is activated, the 
calculation process is executed for calculating target 

20 position coordinates (V 0 , H 0 ) of the extraction position. 
Herein, the target position is given by the following 
equations (1) and (2) with reference to Fig. 18: 



The control steps shown in Fig. 17 are performed 



Referring to Fig. 17, in step S401, determination 



V 0 = V c ± N v =V C ±(-1)- 



f tan9 P 
L v 



(1) 



25 



H 0 = H c ± N v =H C +(-!)• 




(2) 
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where V c and H c denote vertical and horizontal positions 
respectively where a reading out starts when the 
vibration correction is not necessary, 6 P and# y denote 

displacement angles in pith and yaw directions, 
5 respectively, 1^ and L H are vertical and horizontal 
lengths of the CCD, respectively, and N v and N y are 
numbers of shifting pixels for correcting displacement 
angle in the pitch and yaw directions, 

H 0 and V 0 obtained by the above equations (1) and 
*Q 10 (2) are coordinates where the reading-our starts for 

iff anti-vibration control, and thus, is expected to have an 

m integer part and a decimal fragment part. The decimal 

ass 

J fragment part includes important information for 

performing the ant i -vibration control with maintaining a 
U1 15 resolution, and a shift amount represented by the part 

is less than one pixel. The rough pixel shift control 
performed by the circuit 105 is achieved by means of 
reading out of a memory each unit of which corresponds 
to one pixel. The information included in the decimal 
20 fragment part is utilized in the interpolation operation 
by the circuit 105, namely the fine pixel shift control. 

Reading out of the CCD 102 according to the second 
embodiment comprises interlaced readings at even number 
lines and odd number lines which occur every two lines 
25 (pixels in the vertical direction) . Further, the 

readings every two pixels are only made for vertical 
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readings but not for horizontal readings of the CCD 102. 
Therefore, the fine pixel shifting according to the 
second embodiment must be different for in the vertical 
and horizontal directions from each other. 
5 First, the method and scheme for shifting an 

extraction area and a fine pixel shifting in the 
vertical direction will be described below. Letting an 
integer part and decimal part of the shift amount V 0 in 
the vertical direction as Int(V 0 ) and Dec(V 0 ) / 
P 10 respectively, the following equation holds: 

ry 

jn V 0 = Int(V 0 ) + Dec(V 0 ) *•* (3) 

% Where Int(V 0 ) is odd number, and Evn(V 0 ) denotes the 

^ 15 greatest even number which does not exceed the number V 0 , 

it holds : 

S V 0 = Evn(V 0 ) + 1 + Dec(V 0 ) (4) 

20 and where Int(V 0 ) is an even number, it holds: 

V 0 = Evn(V 0 ) + Dec(V 0 ) ••■ (5) 

The second embodiment performs an extraction operation 
25 of the image area accompanied with CCD reading-out as 
illustrated in Fig. 8B, e.g., and an extraction 
operation accompanied with CCD reading-out as 
illustrated in Fig. 8C . In the second embodiment, while 
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the extraction operates is performed, a fine pixel 
shifting by less than one pixel is operated in 
accordance with the value of Dec(V 0 ) for each pixel. 

Described will be the fine pixel shifting in the 
5 vertical direction below. 

In order to initiate the fine pixel shifting, it 
is necessary to determine a distribution ratio K between 
two adjacent pixels in step S404. At first, distribution 
ratio Ky in the vertical direction will be described. In 
10 the second embodiment, the distribution ratio in the 
vertical direction is determined in accordance with 
Dec(V 0 ) or 1+ Dec(V 0 ), and is determined to be as 1/8 or 
7/8. 

As set forth above, CCD reading out according to 

nj 15 the second embodiment is performed at two pixel pitch. 

i y 

ni Thus, fine pixel shifting with shift amount of less than 

/5 one pixel would deteriorate resolution at interpolated 

pixels because the interpolations are made between two 
pixels with two pixel distance apart. The second 
20 embodiment performs interpolations as illustrated in Fig. 
19 in order to achieve both the prevention of unevenness 
in resolution and high resolution. In Fig. 19, the 
vertical axis represent image (pixel position) . 500 to 
503 represent an image signal (signals obtained by the 
25 reading-out as illustrated in Fig. 8B) , 504-506 denote 
image signals with one pixel shift with respect to the 
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signals 500 to 503 (signals obtained by the reading-out 
as illustrated in Fig. 8C) . 600 to 612 are image signals 
generated by interpolating the image signals 500 to 508. 
The interpolated image 600 is generated by a 
5 proportional distribution (ratio 1/8 : 7/8) of the image 
data 500 and 502, and the interpolated image signal 603 
is with proportional distribution ratio 7/8 : 1/8, from 
image signals 500 and 502. On the other hand, the 
interpolated image 604 is generated by a proportional 
^ 10 distribution (ratio 1/8 : 7/8) of the image data 502 and 

|« 503, and the interpolated image signal 608 is with 

U] proportional distribution ratio 7/8 : 1/8, from image 

tn 

HF signals 502 and 503 . 

s Fig. 20 illustrates the image signals 502, 503, 

fjj 15 505, 506, 604, 605, 606 and 607 extracted from Fig. 19 

HI so that a reason why the uneven resolution is prevented 

according to the second embodiment is explained, 

As seen from Fig. 20, pixels of the interpolated 
images (that are output image signals) are apart from 
20 each adjacent pixel by 1/2 pixel distance. In other 
words, the interpolation image has twice as fine in 
space resolving power as original images. The image 
signal 604, e.g. is generated by proportionally 
distributing the image signals 502 and 503 with ratio of 
25 1:7, while the image signal 605 is generated by 

proportionally distributing the image signals 505 and 
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506 with ratio of 7:1. Thus, a resolution at the pixel 
604 is identical with that at the pixel 606, and a 
resolution at the pixel 605 is identical with that at 
the pixel 606. Further, a resolution at the pixel 606 is 
5 identical with that at the pixel 607. Accordingly, the 
pixels 604, 605, 60 6 and 607 are arranged at even 
intervals in the vertical direction, and exhibit the 
same resolution. In other words, there are no uneven 
resolutions between the image signals 604, 605, 606 and 
^ 10 607. More specifically, m the second embodiment, 

arbitrary two subsequent pixels are converted into four 
Ul pixels between which there is no uneven resolution. This 

=p is because all the interpolated pixels are generated on 

5 the basis of the same pair of distribution ratio values 

pi 

ry 15 (pairs of "1" and "7", or pairs of "1/8" and "7/8"). 

py The subsequent two pixels include a pixel on an 

even number line and a pixel at an odd number line, as 
seen from Figs. 8B and 8C. Thus, the value of Int(V 0 ) 
obtained from the equation (3) can indicate which should 
20- be referenced, pixels on an even number line or odd 

number line. Further, since the pitch of interpolated 
pixels is equal to 1/2 (= 0.5) pixel, can be set to 
1/8 or 7/8 depending on whether Dec(V 0 ) is less than or 
greater than 0.5. As set forth above, the value of V 0 - 
25 Evn(V 0 ) can be any value within 0 to 2 . Therefore, in 
order to prevent uneven resolution, it is necessary to 
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determine to be either one of 1/8 or 7/8 by 
quantization . 

Fig. 21 explains a distribution of values of Ky 
according to the second embodiment, and therefore, how 
5 are quantized. In Fig. 21, 700 denotes a position where 
an image extraction starts when Dec(V 0 ) is equal to "0", 
and points 701 to 704 denote a position where an image 
extraction starts when Dec(V 0 ) is unequal to "0". As set 
forth above, the decimal fragment part V 0 - Evn(V 0 ) of 
10 the starting point V 0 can be within the range 0 to 2 . 
Specifically, extraction starting point is coincident 
with: 

point 701 when 0 < V 0 - Evn(V 0 ) ^ 0.5, 
point 702 when 0 . 5 < V 0 - Evn(V 0 ) ^ 1.0, 
15 point 703 when 1.0 < V 0 - Evn(V 0 ) ^ 1.5, 

point 7 04 when 1 . 5 < V 0 - Evn(V 0 ) ^ 2.0. 

As seen from the comparison of Figs 19 and 20, if (V 0 - 
Evn(V 0 ) ) = 0, start point in the vertical direction is 
coincident with the point 700. If 0 < V 0 - Evn(V 0 ) ^ 0.5, 

20 the start point 7 0 1 should be quantized into the pixel 

600, therefore, ¥^ is set to 1/8. In this case, normal 
image data which are not subjected to one pixel shifting 
(image data read-out in accordance with the Fig. 8B 
timing) are used. 
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if (V 0 - Evn(V 0 )) = 0, the start point in the 
vertical direction is set to be coincident with the 
point 700. 

If 0 < V 0 - Evn(V 0 ) ^ 0.5, the start point 7 0 1 

5 should be quantized into the pixel 600 , therefore, is 
set to 1/8. In this case, normal image data which are 
not subjected to one pixel shifting (image data read-out 
in accordance with the Fig. 8B timing) are used. 

^ If 0.5 < V 0 - Evn(V 0 ) ^ 1.0, the start point 7 02 

fy 10 should be quantized into the pixel 603, therefore, is 

Li i 

fps set to 7/8. In this case, normal image data which are 

not subjected to one pixel shifting (image data read-out 
— in accordance with the Fig. 8B timing) are used. 

~ If 1.0 < V 0 - Evn(V 0 ) ^ 1.5, the start point 7 03 

ry 15 should be quantized into the pixel 602, therefore, is 

set to 1/8. In this case, normal image data which are 
subjected to one pixel shifting (image data read-out in 
accordance with the Fig. 8C timing) are used. 

If 1.5 < V 0 - Evn(V 0 ) ^2.0, the start point 704 

20 should be quantized into the pixel 605, therefore, is 
set to 7/8. In this case, normal image data which are 
subjected to one pixel shifting (image data read-out in 
accordance with the Fig. 8c timing) are used. 

The above describes how 1^ is determined in step 

25 S404 according to the second embodiment. 



- 47 - 



Next, the pixel shifting in the horizontal 
direction will be explained. As set forth above, reading 
out of the CCD 102 is performed in accordance with the 
read out sequence as illustrated in Fig. 8B or that 
sequence as illustrated in Fig. 8C . Such modulation 
according to the reading out has effects on only the 
vertical direction. Therefore, in the second embodiment, 
as for the horizontal direction, the switching control 
of readings out of CCD is not performed. Specifically, 



^ 10 since 



H 0 = Int(H 0 ) + Dec(H 0 ) ••• (6) 



holds, positions of interpolated pixels in the 
15 horizontal direction are determined to be intermediate 
between two adjacent pixels in the horizontal direction 
in accordance with the value of Dec(H 0 ). Specifically, if 
[Dec(H 0 )] is an integer which does not exceed Dec(H 0 ), an 
interpolated pixel of n-th and (n+l)-th pixels is set at 
20 the position which is proportionally distributed in 
accordance with the ratio: 

Kh : (1-K H ) 

25 where K H is defined as K H = [Dec(H 0 )] . 

Returning to the flowchart of Fig. 17, in step 
S404, pixel shifting ratio data and 1^ are determined 
based on the decimal fraction part of the target 
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position coordinate data expressed by the above- 
described number of pixels. In practice, pixel-shifting 
data and 1^ may be set to any value from 0 to 1 . It 
is so set that pixel-shifting data or is 1 when 
5 the decimal fraction part is 0. 

In the next step S405 which is the process 
characteristic to the present invention and the details 
are set forth above. The correction variable data is 
regenerated so as to achieve constant resolution on the 
10 screen. In step S406, the target position coordinates 
(V 0 , H 0 ) calculated in step S403 are outputted as a 
command to the CCD driver 106 and the 

enlargement /interpolation/pixel shifting circuit 105 as 
the position to be extracted. In addition, the setting 

15 of color process of the camera signal processor 104 is 

changed in accordance with the setting of the extraction 
position (the camera signal processor 104, 
enlargement/ interpolation/pixel shifting circuit 105, 
and CCD driver 106 operate such that the extraction 

20 position is outputted as commanded in the next field) . 
Then, the process returns to step S401 for processing 
the next field until integration is performed ten times. 

Meanwhile, in the foregoing step S402, if the 
switch 112 is OFF, i.e., anti-vibration function is not 

25 activated, the value for pixel shifting is set to 1 in 

step S407 (not performing pixel shifting) , and a command 
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for extracting the center of the screen and a pixel 
shifting value set in the foregoing step S407 are 
respectively outputted in step S408. 

Why the distribution ratio according to the second 
embodiment is set to 7:1 or 1:7 will be explained below. 

As illustrated in Figs. 19 and 20, the second 
embodiment interpolates two pixels between two adjacent 
pixels so that the resolving power may be doubled. The 
second embodiment necessitates to generate the 
interpolated pixels so that they may have equally spaced. 
In order to achieve the double resolving power as well 
as the equal intervals, an interval between one pixel 
which is normally transferred (like pixel 502) and one 
pixel which is shifted by one-pixel distance (like pixel 
506) have to be divided into divisional ratio 1:2:1. In 
other words, the interval between two pixels which are 
displaced from each other by one pixel distance must be 
divided into four subintervals . In this connection, the 
a pair of two pixels (for instance, the pixels 502 and 
506) cannot be read out simultaneously, as shown in Fig. 
8. For this reason, the second embodiment divides into 
eight (8) sub-intervals, an interval between two pixels 
which are spaced by two pixel distance (pairs of pixels 
each of which are two pixels from the normal image 
signal, such as pixels 502 and 503, or pairs of pixels 
each of which are two pixels from one-pixel-distance- 
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shifted pixel, such as pixels 506 and 507), and then 
proportionally divides the interval with ratio 1/8:7/8. 

As has been described above, according to the 
image sensing method and apparatus of the second 
5 embodiment, pixel shifting operation is performed in 
combination with the operation of changing the pixel 
area normally transferred at one pixel pitch in anti- 
vibration operation. Each of these operations is 
controlled appropriately so as to use a predetermined 
4f 10 correction position only. This attains the effects of 

J™ securing a resolution necessary for anti-vibration, 

keeping the resolution constant on a screen, highly 
HF effective anti-vibration, and a stable screen image. 

fy 15 The present invention is not limited to the above 

fy embodiments and various changes and modifications can be 

made within the spirit and scope of the present 
invention. Therefore, to appraise the public of the 
scope of the present invention, the following claims are 
20 made . 
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